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MISSION OF THE CENTER 



The Center for Vocational and Technical Education, an 
independent unit on The Ohio State University campus, operates 
under a grant from the National Center for Educational Research 
and Development, U.S. Office of Education. It serves a catalytic 
role in establishing consortia to focus on relevant problems in 
vocational and technical education. The Center is comprehensive 
in its commitment and responsibility, multidisciplinary in its 
approach and interinsti tutional in its program. 

The Center's mission is to strengthen the capacity of state 
educational systems to provide effective occupational education 
programs consistent with individual needs and manpower reouire- 
ments by: 

Conducting research and development to fill voids in 
existing knowledge and to develop methods for applying 
knowledge. 

Programmatic focus on state leadership development, voca- 
tional teacher education, curriculum, vocational choice 
and adjustment. 

Stimulating and strengthening the capacity of other agen- 
cies and institutions to create durable solutions to 
significant problems. 

Providing a national information storage, retrieval and 
dissemination system for vocational and technical educa- 
tion through the affiliated ERIC Clearinghouse. 
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FOREWORD 



The project entitled "An Evaluation System for Vocational 
Education Leadership and Professional Development Activities" 
resulted from two generally recognized needs: (l) the need for 

better trained persons in education and (2) the need for better 
evaluation of educational programs. New programs funded with 
money from the 1968 Amendments to the Vocational Education Act 
are beginning to address the first need, although much still needs 
to be done. The need for evaluation of education programs generally 
and of personnel development activities in particular is much more 
apparent. Little systematic study has been conducted to evaluate 
in a meaningful way the training of vocational educators. For 
this reason, the U.S. Office of Education contracted with The 
Center for Vocational and Technical Education to develop an 
evaluation model. 

This report describes the development of a prototype evalua- 
tion model. The Center conducted a pilot test of the evaluation 
model in a subsequent project funded by the U.S. Office of 
Education. 

The services of the following are recognized in completing 
the project: Ronald D. Daugherty, project director, and Warren 

L. Lasell, and Richard P. Coatney. 
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Appreciation is also extended to consultants who assisted 
in the development of the conceptual framework of the model and 
later reacted to a near final draft: Glenn L. Immegart, professor 

of education, the University of Rochester; Robert R. Lange, 
assistant professor. Educational Development, The Ohio State 
University; and John W. Struck, state director of vocational 
education, Pennsylvania. 



Robert E. Taylor 
Director 

The Center for Vocational 
and Technical Education 
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This document is the final report of the project entitled 
"An Evaluation System for Vocational Education Leadership and 
Professional Development Activities." The text of the report 
is arranged in the format for final reports contained in Ap- 
pendix A of the Handbook for Directors of projects funded under 
Parts D and F of the Education Professions Development Act 
(U.S. Office of Education, January 15, 1972). 

1. Restate all of the objectives ( anticipated outcomes ) 
as originally stated in the plan of operation or , if 
applicable , as modified during the operation of the 
project . 

The initial objectives of the project were to develop 
a set of systematic procedures which provide informa- 
tion on a regularly scheduled basis: 

a. To the U.S. Office of Education, state departments 
of education, and institutions of higher education, 
for Tnflking decisions about programs funded under 
Section 552 of Public Law 90-576 

b. To the U.S. Office of Education and state depart- 
ments of education for making decisions about programs 
funded under Section 553 Public Law 90-576 

c. From which to draw conclusions about program focus 
and operation 

d. From which to make recommendations for the improve- 
ment of programs 
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Subsequent to completion of the project, USOE project 
manager. Dr. Lloyd Briggs, modified the original Objectives. 
Following these modifications, the model was designed to 
evaluate the state personnel development programs activities 
funded under Section 553 of Public Law 90-576, as well as 
similar personnel development activities not specifically 
funded by that section of the act. The model was to provide 
guidelines only for individual states to conduct the evalua- 
tion and to utilize this information for future decisions 
regarding similar personnel development activities. 

2 . State the actual outcomes of the project or sub -project 
and describe th e instruments used to measure the outcomes . 

The outcome of the project is the evaluation model attached 
to this report as Appendix A. The outcome of the project 
was measured by reactions of numerous persons in the field 
who are involved in personnel development activities and 
who have expertise in evaluation. They reacted both to the 
conceptualization of the model and the actual procedures 
and materials designed to evaluate personnel development 
programs in the states. Persons who reacted to the model 
at various stages of development are listed in Appendix B. 

3. State the reasons for discrepancies between the anticipated 
and actual outcomes . 

There are no discrepancies between the objectives of the 
project and the outcome. A model to evaluate state personnel 
development programs was produced (Appendix A). The model 
appears to have face validity because of the input and 



reactions of same 22 persons from state departments of 
education and universities. Testing the model was not a 
part of this grant award. The extent, then, to which the 

4 (l . 

model is actually valid and workable was undetermined at 
the completion of the project. 

4. If plans have been made for follow-up evaluation , indicate 
the date when additional information will be available . 

The follow-up evaluation of the model was a pilot test 
conducted under a separate grant in the spring of 1972. 

The purpose of the test was to determine how well the 
evaluation model would work in two states. Information 
was gathered through the use of observation and inter- 
view schedules. This information was used to evaluate 
the model and to make final revisions in it. 

5. State any observations which might be helpful to others 
considering embarking on a venture such as this one . 

a. The fact that the objectives of the project made its 
completion difficult. Objectives for a project such 
as this should be stated clearly and should not be 
changed during the project. 

b. The practice of soliciting numerous reactions from 
many persons several times during the project was 
very helpful in developing a model which was realistic 
for persons in the field. 
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6. State any final recommendations which would he useful 
to the bureau in administering the program under which 
you received your grant . 

The following recommendations are offered to the bureau 
for assistance in administering the program under which 
the grant was given. 

a. Project objectives should be stated clearly and in 
considerable detail in the proposal before the 
project is begun. 

b. The branch should require all states to write dear 
and detailed objectives for their personnel develop- 
ment programs funded by the bureau. 

c. Considerable care should be taken to insure that 
the objectives of a project are not changed during 
the course of the project. 
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The purposes of this overview are (l) to present factors leading 
to the development of the evaluation model for state personnel devel- 
opment in vocational education, ( 2 ) to describe briefly the model, and 
( 3 ) to present a rationale for the model. 

Factors Leading to the Development of the Model 

1. Legislators, government officials, and others increasingly are 
looking to education to solve many of society's problems, and 
they recognize the need for improved personnel in order to achieve 
the solutions. The need for constant updating of knowledge and 
skills is particularly apparent for vocational educators who must 
respond continually to changing manpower needs. 

2. As a result, recent federal legislation has funded programs to 
improve professional persons in education generally and in voca- 
tional education specifically. The money for vocational education 
personnel has been used in part to develop special leadership pro- 
grams leading to a doctorate in vocational education. It has also 
been used to upgrade professional persons in vocational education 
through in-senvce projects. 

3 . Demands for accountability in all educational programs have increased 
Educational decision-makers are being called upon to fix the responsi 
bility for every dollar expended. 

4. Even though there is increased emphasis upon personnel development 
and upon accountability, states generally have expended few, if any, 
resources to evaluate personnel development activities. 
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These factors have led to the funding of a project to develop a 
model for the evaluation of state personnel development programs. The 
model was produced, pilot tested, and revised by The Center for Voca- 
tional and Technical Education, The Ohio State University. 

Description of the Model 

The evaluation model is designed to evaluate state personnel programs 
and projects funded under section 553 of the Vocational Education Amend- 
ments of 1968 and similar projects not funded by that section. In the 
remainder of this document, "program" will refer to the entire state 
personnel development program, and "project" will refer to the activities 
in that program. The following is a description of the components and 
procedures of the evaluation model. 

Components 

The model is comprised of seven major components, as follows. 

1. Model Overview : This document. 

2. State Director's Manual : Provides instructions needed by the 

state director of vocational education to conduct the evaluation. 

3. State Coordinator *s Manual : Contains instructions to assist 

the state coordinator of personnel development in performing 
his functions for the evaluation. 

4. State Evaluation Team Manual : Provides members of the evalua- 

tion team with instructions to prepare an evaluation report 
on the state personnel development program* 
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5. Survey Forms : Used to gather data from (a) the state 

coordinator of personnel development, (b) directors of 
personnel development projects, (c) participants of personnel 
development projects, and (d) supervisors of personnel 
development project participants. 

6. Aid for Interpreting Data : Presents the format for the 

arrangement of data collected with the survey forms and 
also includes questions and instructions designed to assist 
the state evaluation team in preparing the evaluation report. 

7. Guidelines to Develop Behavioral Objectives : Provide assis- 

tance to state personnel in writing behavioral objectives for 
programs and to the state evaluation team in judging per- 
formance objectives. 

Procedures 

The state coordinator of personnel development* (i.e., the person 
designated to administer the state personnel development program) pre- 
pares, distributes, and collects the survey forms. He than arranges 
the data vising the format in the Aid for Interpreting Data (AID). He 
is provided with a computer program which will organize part of the data 
for him. 

The state coordinator next presents the original survey forms and AID 
to the state evaluation team. The state director of vocational education 
has previously selected the team and arranged for its meeting. The team 

* Some states use titles such as part F coordinator and state director of 
personnel development for persons who are responsible for this function. 
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answers questions about the data and collects any additional data needed. 

It then draws conclusions, makes recommendations, and finally, prepares jj 

the final report. This report is presented to the state director. The j 

1 

director reviews the materials with the state personnel development staff j 

and takes appropriate action. The evaluation process is recycled. The 
procedures of the evaluation are shown on the flow chart on the next page. 



Recycle 
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Rationale For The Model 

Reasons follow for: (l) the vise of a state evaluation team, 

(2) the selection procedures for the state evaluation team, ( 3 ) the 
use of a data format and instructions to assist in interpretation of 
data, (4) the packaging strategy of the model, and (5) the use of 
guidelines to develop behavioral objectives. 

Use of an Evaluation Team 

An evaluation team is included in the design as a means to partially 
overcome the problem caused by a lack of agreement on what data are needed 
to evaluate state personnel development activities. Consulted persons in 

state personnel development programs demonstrated little consensus on 

¥ 

data needs. This lack of consensus makes the interpretation of data from 
individual states particularly difficult. As a compromise intended to 
serve as many states as possible, data were chosen to address questions 
related to federal goals for personnel development and several important 
concerns of state personnel development programs that emerged during the 
study. The use of an evaluation team, composed of persons competent in 
several areas and with different perspectives on personnel development, 
is intended to provide as realistic and valid an interpretation of data 
as possible. 

Selection of the Evaluation Team 

The evaluation team should be selected from several different insti- 
tutional levels and should possess some knowledge of vocational education, 
personnel development, and the techniques of data interpretation. Several 
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institutional levels should he represented because both secondary and 
post- secondary institutions are involved in personnel development. The 
team members should possess the knowledge to facilitate utilization of 
the data to prepare an evaluation report. 

The state director of vocational education should select members of 
the evaluation team to insure they are persons who are: (l) knowledgeable 

in both vocational education and evaluation and (2) legitimate to members 
of the State Division of Vocational Education. 

Data Format and Instructions to Interpret 

For three reasons , the state evaluation team will receive information 
arranged in a specified format with questions and instructions to assist in 
the interpretation of data. First, the data collected focus upon the federal 
goals for personnel development in vocational education. Much of the funding 
for programs to be evaluated with this model comes from federal legislation. 
Second, the information collected will cause the evaluation team to concentrate 
upon both state objectives and the degree to which these were attained. 
Finally, the data and instructions will decrease the possibility that the 
state evaluation team will write an overly general report which serves no 
useful purpose. 

Packaging of the Model 

The model is comprised of several materials which may be packaged in 
different groups for convenience. That is, persons who need only certain 
parts of the model may receive only those parts. Those who desire all 
materials, however, may receive them. 
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Use of Guidelines to Develop Behavioral Objectives 

Guidelines to develop behavioral objectives are included to help 
those persons working in state personnel development write specific 
objectives for personnel development programs. Also, the guidelines 
are included to assist evaluation team members in judging the objectives 
supplied to the team with data for the evaluation. 
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Purpose of the Manual 



The purpose of this manual is to provide the State Director of Vocational 
Education with information needed to participate in the evaluation of the 
state program for vocational education personnel development. It is assumed 
that the State Director may want to delegate many of the activities suggested 
in this manual. The manual, therefore, may be used by his staff. It includes 
(l) general procedures for the evaluation ; (2) definitions ; and (3) schedule 
of activities . 

Procedures for the Evaluation 

1. The state will obtain components of the evaluation model. 

(a) An overview of the model, 

(b) The manual for the State Director (this manual), 

(c) The manual for the State Coordinator of Personnel Development, 

(d) Survey forms to collect data from four groups of persons, 

(e) The format for presentation of data, 

(f) The manual for the state evaluation team, and 

(g) The guidelines for writing behavioral objectives. 

2. The state should collect and process information for the evaluation in 
accordance with instructions in the manual for the State Coordinator of 
Personnel Development (M-2) . 
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. The State Director should select five persons to serve on the state 
evaluation team. If feasible, it is suggested that two of the members 
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be from out of state. Members of the team will prepare an evaluation 
report on state personnel development in vocational education - a job 
that will take approximately two days. 

The State Director should choose no more than one person from each of 
the following groups: (a) district or secondary administrators or faculty 

members from a local education agency; (b) the state advisory council for 
vocational education; (c) faculty members in vocational-technical education 
from a university or college within the state; (d) faculty members from a 
university or college within the state (who specialize in research or evaluation) 
(e) the division of research or evaluation within the state department of 
education; or (f) administrators or faculty members of vocational-technical 
education from the community college or technical institute within the state. 

4. The State Director of Vocational Education schedules a meeting of the 
five persons selected to serve on the state evaluation team. The teams 
should meet with the state coordinator of personnel development, an 

ex officio member of the team, in facilities provided by the state. 

5. Following the meeting of the state evaluation team, the team should submit 
an evaluation report on the state personnel development program to the 
State Director of Vocational Education. The report should include 
recommendations concerning the personnel development program in the 
state . 
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Definitions 



Participant — A Person who has participated in a project designed to increase 
his or her effectiveness and efficiency in vocational education. Participants 
may be teachers, coordinators, administrators, or others involved in personnel 
development projects. 

Personnel Development Program — All personnel development projects in the 
state. 

Personnel Development Projects — Components of the state personnel development 
program. The term project, as used herein, does not designate preservice 
teacher education, or preservice education for other positions in the field 
of vocational education. It refers to organized attempts to upgrade professional 
personnel in vocational education such as those funded under section 553 of 
the Education Professions Development Act, but it is not limited to those 
funded under section 553. 



Project Director — The director of a personnel development project, the person 
in charge of the operation of the project. 
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State Coordinator of Personnel Development — The person designated by the 
State Director of Vocational Education to coordinate or supervise the state 
personnel development program. The state coordinator is the person most 
directly responsible for state level personnel development other than preservice 
education. Several states use different terras for this person, for example, 
state director of personnel development and Part F Coordinator. 

25 
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State Evaluation Team — The five-member group designated by the State 
Director of Vocational Education to prepare the evaluation report on state 
personnel development. The team will use data collected with four survey 
forms and supplied by the state coordinator of personnel development to 
prepare its report. 

Supervisor — The immediate supervisor of participants of personnel development 
projects. 

Vocational Education — For the purposes of this evaluation the definition 
of vocational education in the 1968 Amendments to the Vocational Education 
Act is used. 
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Purpose of the Manual 



The purpose of this manual is to provide instructions to assist 
the State Coordinator of Personnel Development for Vocational Education 
in the preparation, distribution, collection, and processing of four 
survey forms. The forms are: (l) the state program coordinator survey, 

(2) the project director survey, (3) the participant survey, and (4) 
the supervisor survey. 

Instructions for Preparation, Distribution, and Collection of Data 

1. The state coordinator should send the project director survey to 

directors of personnel development projects. Suggested steps 

follow. 

a. Make a list of the personnel development projects 
operating in the year. 

b. Assign a number to each project. Assign 1 to the first 
project, 2 to the second project, and so forth. 

c. Place the numbers on the project director surveys in the 
block "For State Use." Place one number on each form. 

d. Check to make sure that each project has the current 
identifying number on its survey form. 

e. Print the title of the project in the space provided 
at the top of the first page. 
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f. Send the project director survey forms to the appropriate 
project directors, with a cover letter. 

g. Also request a prospectus of the project for use by 
the state evaluation team (see instruction 15) . The 
prospectus should include a summary of the project 
(without results) and other descriptive information 
such as the project's length and location, and the 
number of participants . The prospectus can be one that has 
been previously prepared. 

h. The completed project director survey form and prospectus 
should be returned to the state coordinator of personnel 
development. 

2. The state coordinator should obtain lists of participants 
and their supervisors. 

a. Telephone each project director and request an alphabetical 
list of the participants who began the project, whether 
they completed it or not, plus their addresses. 

b. Also, request the names and addresses of the immediate 
supervisors of the participants. To simplify the selection 

of supervisors later, the name of each supervisor should appear 
next to the name(s) of participants who work for the 
supervisor. 

3. The state coordinator completes the state program coordinator 
survey. 



V IV 



29 



After receiving the lists of participants and supervisors, 
the coordinator should determine hew many participants will 
be used from each project. 

a. Record the number of participants in each project. 

b. If the number is over 27, take 20 percent of that 
number (round off to the nearest whole number if 
the result is a fraction). 

c. If the number is 27 or under, always select five partic- 
ipants. (if the project has less than five participants, 
vise everyone in the project.) 

d. The numbers obtained in Steps b and c will be the number 
of participants to be used from each project. An example 
follows instruction 11. 

The coordinator determines which participants within each project 
will be selected. 

a. If there are five or less participants in a project, use 
all participants. 

b. If there are ten or more participants in a project, do 
the following. 

(1) Pick at random any name in the list. 

(2) Starting with the name just below it, count down 
until the fifth name is reached. 



(3) Circle this name. It will he the name of the 
first participant selected. 

(4) Then start with the name just helow the circled 
name and again count down until the fifth name 
is reached. 

(5) Circle this name. It will he the name of the 
second participant selected. 

(6) Count down to the next fifth name in the same manner 
and write it as the third participant selected. 

(7) Repeat this process until the required number of 
names is reached. 

(8) When you reach the bottom of the list, continue at 
the top. 

(9) If you come to a name that has already been selected, 
select the name just below it and continue as before. 

If there are 6-9 participants in a project, use the same 
procedure as in b except each name that is selected should 
be eliminated. Keep eliminating names until the required 
number of participants remains, five (5). 

Use this selection procedure for each project. Remember 
to select five participants for projects of 27 or less, and 
20 percent of the participants for projects of 28 or more. 
An example follows instruction 11. 
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The coordinator should select those supervisors whose 

employees were selected as participants in step 5. 

The coordinator assigns numbers to selected participants 

and supervisors. 

a. Make a list of the sampled participants and their 
supervisors. (It may also be convenient to put the 
addresses in this list in order to facilitate tna-iUng 
surveys to the participants and supervisors.) 

b. Assign a number to each participant. Assign 1 to the 
first participant, 2 to the second participant, and so 
forth. 

c. Keep doing this across projects, so that if the number 
20 is assigned to the last selected participant of 
project 3, then 21 is assigned to the first selected 
participant of project 4 . No two participants should 
have the same number . 

d. Then assign a number to each supervisor in the same 
manner. Assign 1 to the first supervisor, 2 to the 
second supervisor, and so forth. Keep doing this across 
projects. If a supervisor's name appears more than once, 
always give him the same number. For example, if Mr. P. 
T. Jones is assigned 2 and then appears further down the 
list, put another 2 by his name. 

e. The list of selected participants and supervisors, with 
the assigned numbers, should be kept on file. 
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8. The coordinator should assign identifying numbers to the 
participant survey. 

a. Fill out the block at the top of the first page that 
says "For State Use." 

(1) Next to A, write the project number that was used 

to identify the project on the project director survey. 

(2) v Next to B, write the number of the participant's 

supervisor . 

(3) Next to C, write the participant's number. 

b. Print the title of the project in the space provided 
at the top of the first page. 

9. The coordinator collates the pages of the supervisor survey. 

a. Take the first page of the form, entitled "Supervisor 
Survey," and attach it to as many second sheets as there 
are selected participants who work for a supervisor. 

(Each second sheet refers to a participant.) 

b. If a supervisor has only one employee who was selected 
as a participant, attach only one second sheet. 

10. The coordinator assigns identifying numbers to the supervisor 
survey. 

a. At the top of the first page, under "For State Use," 
write the supervisor number next to B. 
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b. At the top of the second page, under "For State Use," 
write the number of the participant's project next to 
A and write the participant number next to C. 

c. Also at the top of the second page, print the name of 
the participant and the title of his project. 

d. Do h and c for each second page of the Supervisor Survey 
Form. 

11. The coordinator mails the Participant Survey, and the 

visor Survey to the appropriate people, with a cover letter. 

A follow-up letter or call may he used to assure a high per- 
centage of returns. Be sure to include a stamped self-addressed 
envelope to those in the sample and to specify the exact project 
to which they are responding. All the forms should he returned 
at a time determined hy the state coordinator and his staff. 
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Example of Determining the Number 
of Participants to be Selected From 
Each Project 

Suppose State Y has a total of 106 participants in five 
projects. The procedure of determining the number in each project 
to be selected is illustrated in the table below. 



Project 


Number 


Number Selected 


1 . 


22 


5 


2. 


6 


5 



3. 


42 


8 (20$ of 42, rounded off) 


4. 


3 


3 


5. 


33 


7 (20$ of 33, rounded off) 




106 





Example of Selecting Participants 
From Each Project 

On the next page are two projects, one with nine participants 
and the other with fifteen. Both projects should have five selected 
(they both have less than 28 participants). For the project with 
nine participants, names are eliminated as illustrated until the 
required number is reached. For the project with fifteen participants, 
names are selected as illustrated until the required number is reached. 
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Example: Selection of Five Participants 

from a Project with Nine Participants 




(Start) : Random 

Selection 
of any name 
in the list 



2nd elimination 
4th elimination 



1st elimination 
3rd elimination 
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Example: Selection of 5 Participants 

from a Project with 15 Participants 



n 

1 . 

2 . 



3. 

4. 

5. 



2 . 

3. 

4. 

5. 
1 . 
2 . 

3. 

4. 



Project 4 

Adams, Carolyn 
Crawford, William 
Dram, Henry 
I Hanson, Burt~l 
I Ho Ison, Teresa I 
Karl, Thomas 
Munn, Nancy 
Neilson, Anne 



(Start): Random 

Selection 
of any name 
on the list 



2nd name selected 
5th name selected 



5. 

1 . 

2 . 

3. 

4. 




( Start ) |i^uison, William"] 

1. Pliny, Curtis 

2. Plotter, Allan 

3. Richter, Warren 

4. Samuels, Richard 

5. 1 Sawyer, Kenneth] 
1. | Talbert, Diane | 



3rd name selected 



1st name selected 
4th name selected 



* Since this name, Kenneth Sawyer, was already chosen, choose 
the name just "below it, Diane Talbert. 
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Instructions for Data Processing 

12. When the state coordinator has received the survey forms, 
he should divide them by project and place them in folders. 

a. Each folder should contain one project director survey 
form and all the participant and supervisor survey forms 
that relate to the project. 

b. Only one form will remain, the state coordinator survey 
form. 

13. The coordinator should place the information from all the 
survey forms into the Format for Presentation of Data. 

a * Place the objectives section of the state coordinator form 
in section A of the Format. Either replace A-l and A -2 with 
the sheets on objectives or take out the objectives infor- 
mation and place it in A-l and A-2. 
b. Place the information from the participant and supervisor 
survey forms in the section of the Format designated with 
the letter B.. 

(l) If the computer program is used, put the information 

on cards according to the instructions with the program. 
The output will give the tables in B-l through B-15. 
Then go to the comments in instruction 9* 
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(2) If the computer program is not used, follow 
instructions 3-8* then go to instruction 9* 

(3) Take a participant form and fill in Tables 1-7 
and the first table on page B-15. Repeat this 
process for each participant form (for Table 7, 
refer to the supervisor form to see at which level 
the participant responses should be placed) . 

(4) Take a supervisor form and fill in Tables 8-l4 
and the second table under additional information. 
Repeat this process for each supervisor form. 

(5) For Tables 1-4 and 8-12, refer to the project 
director survey for information on project size 
and duration. 

(6) In placing the data in B-l through B-15, keep a 
tally of the responses in each cell. You can then 
change the tally marks into numbers and use numbers, 
or you can convert the numbers to percentages. 

(7) Individual projects : If you want one or more tables 

to show the data for individual projects as well as 
the data summed over projects, make as many copies of 
each table as there are projects. Then, as you go 
through a folder, fill in one of the copies. For 
each folder, fill in a copy. While doing this, also 
fill in the original table which shows all of the 
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responses. (For individual projects, only fill in 
the top line of Tables 1-4 and 8-12.) Place the 
individual project information in a separate folder, 
organized by project. Do not place this information 
in the Aid for Interpreting Data. 

(8) For states with few projects : The lines on project 

size and project duration could be used for individual 
projects. For example, if the state has only one pro- 
ject with 1-10 participants, fill out that line under 
project size and you will have individual project infor- 
mation. If all projects are of the same size, you can 
put them on separate lines under project size and cross 
out the project size description in the table. Project 
size information would be of no use anyway. These 
tables (Tables 1-4 and 8-12) may be changed in any way 
that best presents the data. 

(9) Comments : Tear out the third sheet of all the partici- 
pant survey and the second sheet(s) of the supervisor 
surveys. Place these participant comments and supervisor 
comments in the folder that contains individual project 
data. All of this information should be organized by 
project. 
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c. Place the first three pages of the state coordinator 
survey (that dealing with needs) in pages designated 
as C in the Pbrmat (C-l through C-3). Either replace 
C-l, C-2, and C-3 with the three sheets on needs or take 
out the needs information and place it in C-l, C-2, and 
C-3. 

d. Place the last two pages of the state coordinator survey 
(that dealing with selection of projects) in D of the 
Format (D-l through D-3). Either replace D-l, D-2, and 
D-3 with the two sheets on project selection or take out 
the needs information and place it in D-l, D-2, and D-3. 

e. Additional Information : Place the last sheet of the pro- 
ject director survey, the first sheet of each supervisor 
survey, the state plan, and any other information you want 
to include in a separate folder. 

l4. When the information has been placed in the Format, the state 
coordinator should number the pages in the upper right-hand 
corner for easy reference. When the state evaluation team meets, 
give each member a copy of (l) the folder with the Format, with 
data, and the computer output, if any; (2) the folder with project 
information and participant and supervisor comments for each pro- 
ject; and (3) the folder with additional information. States that 
use the computer may, if desired, simply attach the data referring 
to sections A, C, and D to the computer output and place this 
information in a folder (number one above). 
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15. Before the team meets, the coordinator should send members the 
following materials: (l) the overview of the evaluation model; 

(2) the manual for the state evaluation team; (3) a copy of the 
Format for Presentation of Data, with no data; (4) a prospectus 
of each project to he evaluated (see instruction lg on page 2 of 
this manual); and (5) a co sr letter. The cover letter should 
state the purpose of the meeting, the schedule of activities, the 
expected outcomes, and a brief description of the enclosures. 



Vocational Education Personnel Development 
PARTICIPANT SURVEY 



For State Use 
A 

B 

C 



This survey form applies to the most recent personnel development project 
in which you participated. Your responses will be used by the state as one 
indicator of how beneficial personnel development projects (such as inservice 
training) are to professionals throughout the state. Do not put your name 
on this form. 

Project Title 

Please check or complete the following: 

I. BACKGROUND DATA 

1. Age 2. Sex: Male 

Female 



4. Education: No Degree 

Associate Degree 

Baccalaureate Degree 

Masters Degree 

Doctorate Degree 

Others (Specify) 



5. Service Area: Areas in Which Areas in Which 

Training Received Presently Working 

Agriculture 

Business Education 

Distributive Education 

Guidance 

Home Economics ■ 

Technical Education 

Trade and Industry 

Others (Specify) 



II. PROJECT DESCRIPTION 

1. In how many special Vocational Personnel Development Projects or 
activities have you participated? 

(D-4) 



3. Ethnic Identification 
(Optional) 

American Indian 

Black 

Caucasian 

Mexican American 

Oriental 

Other (Specify) 
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Please respond to the remaining items with reference to the most recent 
personnel development project in which you participated. 

2. State what you perceived to he the purpose of the project. 



3* Did you complete the project? Yes 

If no, why not? No 



4. In which type of project or activity 
did you participate? 



Preservice Activity 
Inservice Activity 



5. How long did the project last? 



Less than a week 

One to less than two weeks 

Two to four weeks 

More than four weeks 



Give length of project if more than four weeks: 



III. PERCEPTION OF PROJECT 



Respond to the following dimensions as they apply to the most recent personnel 
development project in which you participated. 

1. Please check appropriately: 



1 

Instructional Activities 


Very 

Worthwhile 


5 

fl 

fa !s 


Not Very 
Worthwhile 


Waste of 
Time 


Does not 
Apply 


Not Available 
but Needed 


a. Buest lecturers and speakers 














b. Use of visual aids 














c. Class discussions 














d. Textbooks and reading materials 














e. Actual or simulated work 
experiences related to 
your job 














f. Others (Please specify) 






• 
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2. How worthwhile was the project for you? 
Comment 



Very worthwhile 
Fairly worthwhile 
Not very worthwhile 
Waste of time 



3. What was most relevant about the project? 



4. What was least relevant about the project? 



5. Did the project apply to your work? 
How? 



Yes 

Partly 

No 



6 . How well organized was the project? 
Comment 



7 . Would you recommend this project to 
someone else? If no, why not? 



Very well organized 
Fairly well organized 
Not very well organized 
Poorly organized 

Yes 

No 



8 . What Vocational and Technical Education Personnel Development 
Projects would you like to see conducted? 



9. Give name and address of employer or immediate supervisor. 



IV. ADDITIONAL COMMENTS 




f 



45 



For State Use 
A 



Vocational Education Personnel Development 
PROJECT DIRECTOR SURVEY 



This survey form applies to the most recent personnel development 
project you have directed. Your responses will be used by the state as 
one indicator of how beneficial personnel development projects (such as 
inservice training) are to professionals throughout the state. 

Please check or complete the following: 

I. BACKGROUND DATA 

No Degree 

Associate Degree 

Baccalaureate Degree 

Master's Degree 

Doctorate Degree 

Others (Specify) 



II. PROJECT DESCRIPTION 

1. State title of project: 



2. At what level was the request for the proposal initiated? 

Federal 

State 

Local 

Other (Specify) 

3. Who had the primary responsibility to develop the proposal? 



2. Was the position of project 
director a full-time or 
part-time assignment? 

Full-time 

Part-time 



(name) 



(position or title) 



4. What was the duration of the project? 



Less than a week 

One to less than two weeks 

Two to four weeks 

More than four weeks 

If more than four weeks, give length of project 

5. What was the total amount budgeted for the project? $ 

6. What percent of the total project budget was allocated for evaluation? 

* 

7* What percent of the total project budget was received from each 
of the following sources? 

Federal Funds % 

State Funds $> 

Local Funds % 

8. What instructional materials or methods were used which provided 
job-like experiences for participants? (e.g., simulated and actual 
classroom experience work in business and industry, etc.)? 



9* List below the educational agencies or units other than the State 
Division of Vocational Education that provided support for this 
personnel development project. Give the type of activities 
performed by these agencies or units. 



Educational Agencies 
of Units 



Type of Activity 
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What Vocational and Technical Education Personnel Development 
Project would you like to see conducted? 



11 • Please list below each stated objective of the personnel 

development project and give evidence of attainment. If the 
evidence is in a document, please reference the document 
attach a copy of it to this instrument. If there are more than 
three objectives, attach additional sheet. 

Objective 1. 



Evidence of attainment: 



Objective 2. 



Evidence of attainment: 



Objective 3. 



Evidence of attainment: 



12. ADDITIONAL COMMENTS 



Vocational Education Personnel Development 
SUPERVISOR SURVEY 



For State Use 
B 



This survey form applies to the personnel development project(s) in 
which one or more of your employees recently participated. Please complete 
the first page of the form only once. Complete a second page for each of 
your employees who were in personnel development projects (at the top of each 
second page is the name of your employee). Your responses will he used by 
the state as one indicator of how beneficial personnel development projects 
(such as inservice training) are to professionals throughout the state* 

Do not put your name on this form. 

Please check or complete the following: 

I. DESCRIPTIVE INFORMATION 

1. What is your title? 



2 . 



With which of the following agencies are you associated? 

College or University 

Community College or Technical Institute 

Intermediate School District 

Local School District 

State Department 

Others (Specify) 



3. 



Local School District: (Complete if you are associated with) 

a. Size of school district: Less than 10,000 pupils 

10,000 - 20,000 pupils 

20,001 - 50,000 pupils 

More than 50,000 pupils 



b. 



Average family income for 
total school district 
population: 



Less than $3,000 
$3,000 - $6,000 
$6,001 - $10,000 
$ 10,001 - $ 15,000 
$ 15,001 - $ 20,000 
More than $20,000 



C# 



What percent of the schools in the district are actually 
receiving Title I funds? 



d. What percent of the student body is receiving ADC? 



4. If you are associated with a College or University, a Community 
College or Technical Institute, what percent of the student body 
is receiving financial aid? $ 

II. VOCATIONAL DEVEL0R4ENT NEEDS 

What Vocational and Technical Education Personnel Development Projects 
would you like to see conducted? (Attach sheet if space is needed. ) 



(D-5) 
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Employee’s (Participant's) Name 

Project Title 

III. PERCEPTION OF PROJECT 



For State Use 

A 

B 



1. Did the project apply to participant’s work? 

Yes 

Partly 

No 

2. Has there been a noticeable improvement in the job performance 
of participant after participating in the project? If so, how 
has he improved? 

Great improvement 

Some improvement 

No improvement 

3. Would you encourage other employees of your agency to participate 
in a similar project? Please explain. 

Yes 

No 

4 . Has participant shared his experiences with you, his immediate 
supervisor? 

Yes 

No 



5. Has participant shared his experiences with other employees 
with whom he works? 



Yes 

No 

Do not know 



6. What provisions were made for participant to share his experiences? 



Group seminars 
Newsletters 
Informal discussions 
Others (Specify) 



IV. ADDITIONAL COMMENTS 






T 
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Vocational Education Personnel Development 
; STATE PROGRAM COORDINATOR SURVEY 

j I. DETERMINATION OF PERSONNEL DEVELOPMENT NEEDS. 

i 

Please complete the following table by ranking the means of determining 
| personnel development needs in order of importance (l being most 

! important). The means of determining needs are lettered A through 

j E (A is consultation, E is other). 

i 

I 



Order of 
Importance 


Means of Determining Needs 




! 


Consultant with persons in the 
agencies listed. Circle appro- 


With Whom Consulted 
(Title and/or Functions) 






priate letters. 

a. Local Education Agencies 

b. State Education Agencies 

c. Universities 

d. Community Agencies 

e. Business and Industrial 
Organizations 

f . Labor Organizations 

g. State Personnel Directors 

h. State Advisory Council for 
Vocational Education 

i. Other (Specify) 




> 

i 




1 

I 


j 


Review of personnel development literature 
List Sources: 








Needs generated by previous and/or ongoing projects within 
your State. List Projects: 




i 


Your own observation and/or experience. 
Explain: 






j 


Other (Specify). 












(D-2) 
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i 

1 

i 

! 

t 

: 



f 

t 



i 



II. NEEDS AND PRIORITIES 

The personnel development program under your direction consists 

of projects and activities designed to address specific needs. 

DIRECTIONS: 

A. In Section I of the chart below, please list the stated personnel 
development needs in order of priority. (See example of a need 
below) . 

B. Section II consists of numerous criteria usually considered in 
setting the priority of needs. For each need listed in Section 
1, rank in order of importance the criteria having the greatest 
influence in establishing its priority, (l being the most im- 
portant). See example below. 

C. Check appropriately, Section III, if adequate resources are 
available to conduct projects and/or activities designed to 
address the stated need. 

NOTE: If additional space is required, continue on page 2a. 



SECTION I 


SECTION II 


SECTION 


III 




Personnel Development 


i 


Criteria for Setting Criteria 


Adequate 




Needs Listed in Order 




Opinions of 
State Voc. Ed 
Personnel 


Public 


Existing or 


Advice of 




Other 

Sources 

(Specify) 


Resources 




of Priority 


Resources 

(Plnanclal, 

Personnel) 


Policy 

1 Federal, 
State) 


Projected 
Shortage of 
Personnel 


8tate 

Advisory 

Council 


Mansger 

Ability 


Yes 


No 


Inservice Training 
for 50 Local School 
Voc. & Tech. Educa- 
tion Curriculum 
Specialists 




5 


i 


4 




2 


Local 

School 

Supt. 

3 
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SECTION I 

Personnel Development 


SECTION TT 

Criteria for Setting Priority 


SECTION III 
Adequate 

Rocnnrooc 


Needs Listed in Order 
of Priority 






Public 

Policy 

(Federal* 

State) 


existing or J 
Projected ( 
Shortage of i 
Personnel ( 


Id vice of 
State 
Idvlsoiy 
Council 










Resources 

(Financial, 

Personnel) 


Opinions of 
State Vos« Ed 
Personnel 


Other 

Sources 

(specify 


Manager 
j Ability 


Yes 


No 




















s 



\ D. Because of various constraints, for example, lack of available 
resources, state regulations etc. , certain needs may be reduced 
in priority. State the reduced priority needs and those con- 
1 straints «hich keep the priority low. 

! 
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i III. OBJECTIVES OF PERSONNEL DEVELOPMENT PROGRAM 

j 

| A. Personnel Development Programs are designed to meet a number 

j of specified objectives. Please list the objectives of the 

j Personnel Development Program in your state. For each objective, 

i present the evidence of accomplishment for that objective. If 

j the evidence is in a document, please reference the document and 

attach a copy of it to this instrument. Attach additional sheets 
if necessary. 



B. Are there projects which do not address program objectives? 

Yes 

No 

If Yes, give nature of project (s) and basis of its priority. 
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IV. RESPONSIBILITY FOR PERSONNEL DEVELOPMENT IN STATE 

Give name and title of the following: 

1. Persons directly responsible for personnel development program 
in state. 

(T'.tle) 

2. Person who has control of the allocation of funds for vocational 
education personnel development in the state. 

(Title) 

3. If 1 and 2 are different people, is there a resulting difficulty 
in funding personnel development projects in the state? 

I | YES 1 I NO 

Please explain: 



V. PROCEDURES FOR SOLICITING PROPOSALS 

A. What percentage of the proposals for projects designed to meet the 
program objectives were from the following organizations or agencies: 



Agencies 


Percent of 
Total Proposals 
Solicited 


Percent of 
Total Proposals 
Submitted 


Percent of 
Total Proposals 
Funded 


Universities 








Local education agencies 








Community or junior college 








Private R & D agencies 








State education agencies 








Other ( Specify i 
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B. List the steps used to review proposals for Personnel 
Development Projects. 



Steps in 
the Review 


People Involved 
(Title and/or Function) 







C. What person or group by title or function, makes the final 
decision to fund a given personnel development project? 



VI. CRITERIA. FOR SCREENING PROPOSALS 

Check the criteria used over the total process of screening proposals 
(Section II) and rank in orderof importance (Section III) (1 being 
the most important). 

NOTE: If criteria varies by program objectives, reproduce chart 

for each objective. 



SECTION I 


SECTION II 


SECTION HI 


Screening 

Criteria 


Criteria 

Used 


Rank 


Relationship to state program objectives 






Estimated cost of proposal project 






Number of students (participants) involved 






Students (from participant's school or 
agency for whom the project is aimed 






Past record of institution or agency 
(success with previous projects) 






Proposed project duration 






Feasibility of projects 






Ability and availability of personnel 






Other criteria used (Specify) 
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VII. EXPENDITURE OF RESOURCES 

A. What was the total cost of the program? 



B. What were the administrative costs of the program? 

C . What percent of the total program budget was allocated 

for evaluation? 

D. List below the educational agencies or units other than the 
State Division of Vocational Education that provided support for 
the personnel development effort within your state. Give types 
of activities performed by the agency or unit. 



Educational Agency 
or Unit 


Type of Activity Performed 







VIII . ADDITIONAL COMMENTS : 



O 

ERIC 
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A MODEL TO EVALUATE 
PERSONNEL DEVELOPMENT IN 
VOCATIONAL-TECHNICAL EDUCATION 



Manual for the 
State Evaluation Team 



Warren L. Las ell 
Richard P. Coatney 
Ronald D. Daugherty 



Draft for Pilot Test 
January 15 * 1972 



The purpose of this manual is to assist members of the state evalua- 



tion team in evaluating their state's personnel development program 
in vocational-technical education. It provides guidelines and/or instruc - 
tions for: (l) planning the state evaluation team meeting and (2) pre- 

paring an evaluation report chi the state personnel development program 
for the state director of vocational education. The manual should he 
used with the Aid for Interpreting Data (AID) to assist the state evalua- 
tion team in preparing an evaluation report. 

Guidelines and Instructions 

The state director of vocational education should select the five 
members of the state evaluation team (SET) and schedule the initial 
meeting of members and the state coordinator of personnel development 
(an ex officio member). Suggested instructions follow for planning the 
meeting and preparing a report. 

1. Plan the team meeting. The following steps are su gg ested 
as possible planning functions for the team, 
a. Determine the nature and scope of the team's job . The 
team may use the overview of the evaluation model, this 
manual, and other documents provided by the state 
coordinator of personnel development to gain a better 
understanding of the necessary functions of the evaluation 
team. 
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t>. Determine the resources and time available to perform the 
evaluation of the state personnel development program . 

Although team members may request more information from the 
state coordinator of personnel development, they should 
receive from him as a minimum: 

(1) The overview of the model, 

(2) The aid for interpreting data /AID will have informa- 
tion supplied from the four survey forms listed below 
in (3) through (6J7, 

(3) The state coordinator survey, 

(4) The project director survey, 

(5) The participant survey, and 

(6) The supervisor survey. 

c. Determine additional Information needed . The team may 
want additional data to evaluate the state personnel 
development program. For example, the questions from AID 
which are designed to assist team members may suggest the need 
for additional, data which could he found in the state plan 
for vocational education. 

d. Organize the team for the evaluation . The team should 
designate a chairman to moderate meetings and to present 
the evaluation report to the state director of vocational 
education. Members should also make any other organizational 
arrangements they feel will facilitate the evaluation. 
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2. Prepare the evaluation report. 

a. Make sure that each SET member has: 

(1) Aid for Interpreting Data 'which includes questions 
to assist the team in preparing an evaluation report, 

(2) Data collected with: (a) the state coordinator survey, 

(b) the project director survey, (c) the participant 
survey, and (d) the supervisor survey (presented in 
the format of AID), and 

(3) Other data which members of the state evaluation team 
feel are necessary to answer the questions included 
with the classification scheme or to prepare a report. 
(The additional data would be supplied by the state 
coordinator of personnel development). 

b. Use the analysis questions and data to write the evaluation 
report. 

c. The report should include: 

(1) An introduction which cites the purpose of the evaluation 
report and provides background of the report, 

(2) A main body of the report which contains: 

(a) Questions examined by the state evaluation team, 

(b) Information presented to answer the questions, and 

(c) A discussion of the information; 

(3) A summary of the findings and conclusions of the evalua- 
tion team, 

(4) Recommendations of the team. 
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A MODEL TO EVALUATE 
PERSONNEL DEVELOPMENT IN 
VOCATIONAL-TECHNICAL EDUCATION 



Aid for Interpreting Data 



Warren L. Lasell 
Richard P. Coatney 
Ronald D. Daugherty 



Draft for Pilot Test 
January 15 , 1972 



| The purpose of the Aid for Interpreting Data (AID) is to present 

information to members of the state evaluation team (SET) in a systematic 
■ way. AID is an outline, the first section of which consists of major 

topics, survey items, and information. Its second section is made up of 
questions designed to assist in the interpretation of the information. 

Four main topics, developed as a result of the study of state and 
: federal vocational personnel development programs, are presented in the 

j first section of AID . These are: (A) Program Objectives, (B) Participant 

and Supervisor Opinions, (C) Personnel Development Needs, and (D) Selection 
1 of Projects. Under each of the four topics, items from survey forms and 
information related to the item are presented. In some cases the informa- 

\ 

l tion is a direct response to the item by respondents; in others, the informa- 

\ 

| tion is an analysis of data, such as a table with data from several survey 

i 

\ items . 

| Pages in the first section are designated with the letters next to 

! the topics in the above paragraph; that is. Program Objectives are 
| designated as A, Participant and Supervisor Opinions, B, and so forth. 

[ They are numbered from 1 through the number needed. For example, 

| 

\ materials under Program Objectives are designated A-l and A-2. 

5 



O 
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The second section of AID presents questions and directions designed 



to assist team members in interpreting data collected with the survey forms . 
Questions are written on perforated pages and are referenced to pages in the 
first section which have information to which the questions relate. Indi- 
vidual pages from Section II may be removed and placed next to information 
in Section I to facilitate the interpretation of data. 

Caution 

Interpretations of the data must be made carefully. Data are indicators 
only and, by themselves, prove nothing. Therefore, SET members should look 
for trends and attempt to relate different pieces of information. 



A-l 



I. SURVEY ITEMS AMD INFORMATION 
Objectives of the State Program 



Survey Item : 

Personnel Development Programs are designed to meet a number of 
specified objectives. Please list the objectives of the Personnel 
Development Program in your state. For each objective, present the 
evidence of accomplishment for that objective. If the evidence is 
in a document, please reference the document and attach a copy of 
it to this instrument. Attach additional sheets if necessary. 



Information: 
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Participant and Supervisor 
Opinions of Programs 

This section includes l4 tables and two questions asked of participants. 
Information in the tables comes from several survey items. No survey items per 
se, therefore, are on listed pages with tables . All l4 tables are listed under 
one heading far information. The next heading designating a survey item does 
not appear again until page B-l6. 

When interpreting information in the tables, team members should pay 
particular attention to the number of responses and non-responses. A 
small number of responses or, conversely, a large number of non-responses 
requires caution . In either case extreme caution should be taken before 
generalizations are made. 



j 

jt 

\ 

i 

| 
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Information: 



TABLE 1 



Participant Reaction to "How Worthwhile Was the Project?": 
Over All Projects, By Project Size 
and By Project Duration 



Projects 


How Worthwhile Was the Project? 


Participants 


Very 


Fairly 


Not 

Very 


Waste of 
Time 


No 

Response 


Number 


t 


All Projects 














100 


Project 
Size (by 
number of 
partici- 
pants ) 


1-10 
















11-20 
















21-30 
















Over 

30 
















Project 
Duration 
(in weeks) 


Less than 
1 
















1 less 
than 2 
















2-4 
















More 
than 4 
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TABLE 2 



Participant Reaction to "Did the Project Apply to Work?": 
Over All Projects, By Project Size 
and By Project Duration 



Projects 


Did the Project Apply 
to Work? 


Participants 


Yes 


No 


Not 

Responding 


Number 


4 


All Projects 










100 


Project 
Size (by 
number of 
partici- 
pants ) 


1-10 












11-20 












21-30 












Over 

30 












Project 

Duration 

(in 

weeks) 


Less 
than 1 












1 less 
than 2 












2-4 
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TABLE 3 



Participant Reaction to "How Well Organized Was the Project?": 
Over All Projects, By Project Size 
and By Project Duration 



Projects 


How Well Organized Was the Project? 


Participants 


Very 

Well 


Fairly 

Well 


Not Very 
Well 


Poorly 


Not 

Responding 


Number 


i 


All Projects 














100 


Project 
Size (by 
number of 
partici- 
pants) 


1-10 
















11-20 
















21-30 
















Over 

30 
















Pro j ect 

Duration 

(in 

weeks) 


Less than 
1 
















1 less 
than 2 
















2-4 
















More 
than 4 
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TABLE 4 

Participant Reaction to "Would Participant Recommend 
Project to Someone Else?": Over All Projects, By 

Project Size, and By Project Duration 



Projects 


Would Participant 
Recommend Project to 
Someone Else? 


Participants 


Yes 


No 


Not 

Responding 


Number 


$ 


All Projects 










100 


Project 
Size (by 
number of 
partici- 
pants) 


1-10 












11-20 












21-30 












Over 

30 












Project 

Duration 

(in 

weeks) 


Less 
than 1 












1 less 
than 2 




- 








2-4 
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TABLE 7 

Participant Reaction to "How Worthwhile was 
"Did Project Apply to Work?" By Thr 
Measures- of LEA of Participa: 




No 

Response 
































■ 


How Worthwhile was the Project?, 


Waste 
of Time 


































Not 

Very 


































1 

Fairly | 
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TABLE 8 



Supervisor Reaction to "Did Supervisor Feel Project Applied 
to Participant’s Work?" : Over all Projects, by 

Project Size, and by Project Duration 



Projects 


Did Supervisor feel Project 
to Participant's Work 


Applied 


Supervisors 


Yes 


Partly 


No 


No 

Response 


# 


% 


All Projects 












100 


Project Size 
(by number of 
Participants) 


1-10 














11-20 














21-30 














Over 

30 














Project 
Duration 
(in weeks) 


Less 
them 1 














1-less 
than 2 














, 2-4 














More. 

than4 
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TABLE 9 



Supervisor Reaction to "Has there been Improvement in Performance 
of Participants?"; Over all Projects, by 
Project Size, and by Project Duration 







Has there been Improvement in 
Performance of Participants? 


Supervisors 


Projects 




Great 


Some 


No 


No 

Response 


# 


1 


All Projects 












100 


Project Size 


1-10 














(by number 


11-20 














of partici- 


21-30 














pants) 


Over 

30 














Project 


Less 
than 1 














Duration 


1-less 
than 2 














(in weeks) 


2-4 














More 
than 4 















7fi 



TABLE 10 



\ 

Supervisor Reaction to "Has Supervisor Encouraged Other Employees 
x to Participate in Similar Projects?": Over all Projects, 
by Project Size, and by Project Duration 



Projects 




Has Su] 
Other Ei 
pate in 


pervisor Encouraged 
nployees to Partici- 
Similar Projects? 


Supervisors 






Yes 


No 


No 

Response 


# 


i 


All Projects 










100 


Project Size 


1-10 












(by number of 


11-20 












participants) 


21-30 














Over 

30 












Project 


Less 
than 1 












Duration 


1 less 
than 2 












(in weeks) 


2-4 














More 
than 4 













V 



7 



1 
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TABLE 11 



Supervisor Reaction to "Has Participant Shared his Experiences with You, 
his Immediate Supervisor?": over all Projects, by 

Project Size, and by Project Duration 



i i 







Has Participant shared 
Experience with Supervisor? 


Supervisors 


Projects 




Yes 


No 


No 

Response 


# 




All Projects 










100 


Project Size 


1-10 












(by number of 


11-20 












participants] 


21-30 














Over 

30 












Project 


Less 
than 1 












Duration 


1 less 
than 2 












(in weeks) 


.2-4 














More 
than 4 













i 

f 



ERIC I 



78 



TABLE 12 



Supervisor Reaction to "Has Participant Shared Experiences 
With Other Employees?": Over All Projects, 

By Project Size and By Project Duration 



Projects 


Has Participant 
Shared Experiences 
with Other Employees 


Supervisors 


Yes 


No 


No 

Response 


Number 


i 


All Projects 










100 


Project 

Size 

(by number 
of partic- 
ipants ) 


1-10 












11-20 












21-30 












Over 

30 












Project 
Duration 
(in weeks) 


Less than 
than 1 












1 less 
than 2 












2-4 












More 
than 4 













Supervisor Reaction to "Did Project Apply to Participant's Work 
and "Has There Been Improvement in Performance of Participant? 
Ry Three Economic Measures of LEA of Participants 
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b-i4 
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Supervisor Reactions to "Did Project Apply to Participant’s Work? 
and "Has there been Improvement in Performance of Participant?" 
by Size of LEA and Educational Agency Supervisors 



B-15 




Supervisors 




























** 


























g 

♦H C** 
W 


No 

Response 


























Has there been Improvement 
Performance of Participant 


si 


























Some 


























Great 


























Did Project A - , ply to Participant's 
W' rk? 


No 

Response 


























!§ 


























Partly 


























Yes 


























V 

. g 
'd <u 
C tiD 

3 < 


Less than 10,000 


0 1 

CVJ 

1 

2 
o 

9 


0 

in 

1 

i 

* 

o 

cu 


More than 50,000 


College or 
University 


Community College 
or 

Technical Institute 


Intermediate 
School District 


Local School 
District 
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Survey Item : 



What was most relevant about the project? 



Information: 



O 

ERIC 






Survey Item : 



What was least relevant about the project? 



Information: 



C-l 



Personnel Development Needs 



Survey Item with Information 

Please complete the following table by ranking the means of 
determining personnel development needs in order of importance 
(.1 being most important). The means of determining needs axe 
lettered A through E (A is consultation, E is other). 



j Order of 
i Importance 


Means of Determining 


Needs 


U* 


Consultant with persons in the agencies 
listed. Circle appropriate letters. 


With Whom Consulted 
(Title and/or Functions) 


f 

I 

! 

t 

i 

f 

i: 

i 

r 

/' 

\ 

k 

r- 

t 

? 

r 

[• 

f 


a. Local Education Agencies 

b. State Education Agencies 

c. Universities 

d. Community Agencies 

e. Business & Industrial 
Organizations 

f. Labor Organizations 

g. State Personnel Directors 

h. State Advisory Council for 
Vocational Education 

i. Other (Specify) 




i 

J. 

s 

i B- 


Review of personnel development literature 
List Sources: 




l 

5 

f 

f 

i 

i 

1 r. 


Needs generated by previous and/or ongoing projects within your 
State. List Projects: 


D. 


Your own observation and/or experience. 
Explain: 




E. 


Other (Specify). 
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Survey Item and Information 
DIRECTIONS: 



A. In Section I of the chart below, please list the stated 
personnel development needs in order of priority. (See 
example of a need below). 



i 

l 

l 

j 

i 

i 

i 

I 

t 

f 

I 



B. Section II consists of numerous criteria usually considered 
in setting the priority of needs. For each need listed in 
Section I, rank in order of importance the criteria having 
the greatest influence in establishing its priority, (l 
being the most important). See example below. 

C. Check appropriately, Section III, if adequate resources are 
available to conduct projects and/or activities designed to 
address the stated need. 



( 



SECTION I 



Personnel Development 
Needs Listed in Order 
of Priority 



Resources 

(rinancltl 

Personnel) 



Inservice Training 
for 50 Local School 
Voc. & Tech. Educa- 
tion Curriculum 
Specialists 



SECTION II 



Criteria for Setting Priority 



Opinions of 
Slat* Voc* Ed. 
Personnel 



Public 
Policy 
(Federal, 
OtaU) 



touting or 
Projected 
Ghort*e« Of 
Personnel 



Mvles of 
State 
AdvUory 
Council 



Himii* 

eWIltf 



iXher 

Sources 

(Gpteify) 



SECTION m 

Adequate 

ResourccD 



Yes 



No 



5 



1 



4 



2 



Local 

School 




Supt. 

3 . 



O ! 

ERJCl 



S5 
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j Survey Item 

| 

; Because of various constraints, for example, lack of available 

! resources, state regulations, etc,, certain needs may be reduced in 

! priority. State the reduced priority needs and those constraints 

; v/hich keep the priority low. 



j Information : 

i 

J 



r 

t 
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D-l 



Selection of Projects 



Survey Item and Information 

How many and what percentage of the proposals for projects designed 
to meet the program objectives were from the following organizations or 
agencies: 



Agencies 


Percent of Total 
Proposals Submitted 


Percent of Total 
Proposals Funded 


Universities 


No. 


% 


No. 












Local education agencies 










Community or junior college 










Private R & D agencies 










State education agencies 










Other (Specify) 































Survey Time and Information 

List the steps used to review proposals for Personnel Development 
Projects. 



Steps in the Review 


People Involved 
(Title and/or Function) 







D-3 



Survey Item and Information 

Check the criteria used over the total process of screening 
proposals (Section II) and rank in order of importance (Section III) 
(1 being the most important). 

NOTE: If criteria varies by program objectives, 

reproduce chart for each objective. 



SECTION I 


SECTION II 


SECTION III 


Screening Criteria 


Criteria 

Used 


Rank 


Relationship to state program objectives 






Estimated cost of proposal project 






Number of students (participants) involved 






Students (from participant's school or 
agency) for whom the project is aimed 






Past record of institution or agency 
(success with previous projects) 






Proposed project duration 






Feasibility of projects 






Ability and availability of personnel 






Other criteria used (Specify) 
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II. QUESTIONS AND INSTRUCTIONS 



Underlined headings are used to designate the pages in Section I of 
Aid for Interpreting Data where data to which the questions and instruc- 
tions refer. The pages which follow may he removed so that the questions 
and instructions may he viewed next to the information in Section I to 
which they relate. 



A-l 



1. Based upon the evidence presented rate each of the state program 
objectives according to the scale which follows. 

2 3 4 



partially fully 

attained attained 



not 

attained 



2. For each objective, state the reasons for the rating given. 

3* Refer to information on the state needs for personnel development 
(C-l, ff.) Are the objectives consistent with those needs? 

4. What are the major assets which contributed to attainment of 
objectives? 

5. What were the major obstacles to the attainment of objectives? 

objectives stated as behavioral objectives? (see the Guideline 
for Developing Behavioral Objectives ) . How can objectives which are 
not behavioral objectives be improved? 

7. Is there additional information to determine the attainment of objec- 
tives? If so, what information? 

8. What conclusions about the state personnel development program can 
be drawn? 



1. Is there justification for the projects which do not address program 
objectives? 

2. What conclusions can be drawn from the data? 
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B-2 through B-15 

The paragraphs which follow are designated: (l) analysis, (2) cau- 

tions, (3) conclusions, and (4) individual projects. They include instruc- 
tions to assist SET members in analyzing and reaching conclusions, as w ell 
as cautions which should be observed in interpreting the data. 

Analysis , Cautions, and Conclusions : Tables 1-14 

1. Analysis: 

(a) For each level of a table (i.e., each horizontal row), 
compare the percentage of favorable and unfavorable 
participant ratings. For Tables with Yes-No or Yes-Partly- 
No ratings, simply compare the percentage of Yes to the 
percentage of No responses. For Tables with a Worthwhile 
rating, compare the sum of the very and fairly worth- 
while ratings to the sura of not very worthwhile and 
waste of time ratings. For Table 9 > compare the sum 

of the Great and Some percentages to- the No percentage. 

(b) For Tables 1, 3, 5, 6, and 7, does any extreme percentage 
(i.e., completely worthwhile and not worthwhile) seem large 
enough or small enough to bear mentioning? For Tables 6, 

13, and 14, does the percentage of Great responses seem 
large enough or small enough to bear mentioning? Report 
these percentages and the level of each percentage. 

(c) For Tables 1-4 and 8-12, refer to the survey forms with raw 
data. Read through the comments under the question that 
relates to a table. For example, on Table 1 read through 
the comments on the participant survey form under "How 
Worthwhile was the Project?" List comments that you feel 
are significant, if any. 

(d) For Table 5, under each activity, look at the percentage of 
participants who said the activity is not used but needed. 
Does any percentage seem large enough to bear mentioning? 

(e) For Table 6, look at the percentage of participants. For 
age and for ethnic group, does any percentage stand our as 
very high or very low? For sex, does either sex have many 
more participants than the other? Report any significant 
percentage and the group characteristics associated with it. 

2 . Cautions : 

(a) For all tables except Table 5> does the percentage of No 
responses for each level seem high enough to make inter- 
pretations of that level difficult? For Table 5, does the 
sum of the last three percentages of each level seem high 
enough to make an interpretation of the level difficult? For 
Tables 1-4 and 8-12, does the percentage of No responses for 



SO 



II-3 



the 'All Projects " level seem high enough to make an inter- 
pretation of the entire table difficult? Report all levels 
or tables that seem difficult to interpret and the percen- 
tage of No responses of each. 



(b) For all tables except Table 5, does the number of partici- 
pants for each level seem low enough to make an interpre- 
tation of that level difficult? Report the levels that seem 
difficult to interpret and the number of participants in 
each. (Question 3 refers to the situation where there are 
no participants in a level; do not refer to that situation 
here). For Table 6, is the number of participants who did 
not identify their ethnic group high enough to make an in- 
terpretation of ethnic groups difficult? Report this number 
and the percentage; if too high, do not interpret ethnic 
group data at all . 

(c) For those tables with a Yes-Partly-No rating, does the per- 
centage of Partly responses seem so high that a comparison 
of favorable to unfavorable for a particular level seem 
difficult? If so, report the level and the percentage of 
Partly responses. 

3* Conclusions: 

Base your conclusions on the data examined in 1 above. Keep in 
mind the cautions brought out in number 2. Because of the cau- 
tions, you may decide not to include a certain level in your 
conclusions or not to make conclusions at all. You may want to 
comment on decisions you make. If a level has no participants, 
this level will be excluded from your conclusions, unless you 
have a comment about it. You may notify any of these instructions 
or you may organize your conclusions differently if you feel such 
changes are best for the program you are evaluating. For Tables 
1-4 and 8-12, make conclusions about : 

(a) , .participant or supervisor views over all projects 

(b) participant or supervisor views according to the 
size of projects 

(c) participants or supervisor views according to the 
duration of projects 

(d) participant or supervisor comments that were not 
brought out in a, b, or c, 

(e) any other aspect of the data. 

For Table 5, make conclusions about : 

(a) participant views of the worthwhileness of each activity 

(b) participant views of the need for each unused activity, 
if there are sufficient data 

(c) any other aspects of the data. 
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For Tables 6, 7» 13 » and 14, make conclusions about : 

(a) participant or supervisor views under each of 

the two questions, according to each characteristic. 
(For example, in Table 6, participant views of 
worthwhileness and applicability, according to 
age, sex, and ethnic group). 

(b) For only Table 6, the distribution of participants 
by age, by sex, and by ethnic group, 

(c) Any other aspect of the data. 

4. Individual Projects : 

If Tables for individual projects are examined, do any of the 
projects seem very much different from the rest, either pos- 
itively or negatively? Identify these projects, with supporting 
data. From the answer to this question, does it seem that your 
conclusions about the state program are effected equally by 
all projects or predominantly by a few. 



B-l6 



1. What responses were most common? 

2. How do the responses to this item compare with the information in 
Tables 1 and 2 (pp. B-2, B-3) which show the reactions of participants 
to "How Worthwhile was the Project?" and "Did the project apply to 
Work?" 

3. What conclusions can be drawn from the information and a comparison 
with other data from participants? 



B-17 



1. What responses were most common? 

2. How do the responses to this item compare with the information in 
Tables 1 and 2 (pp. B-2, B-3) which show the reactions of participants 
to "How Worthwhile was the Project?" and "Did the Project Apply to 
Work?" 

3. What conclusions can be drawn from the information and a comparison 
of it with the other data from participants? 
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C-l 

1. What methods were used to determine the personnel development needs? 

2. What method was rated as most important? 

3. How many persons were consulted? 

4. In you r opinion, were there other means which might have been used 
to determine personnel development needs? 

5. Was the process used to determine needs consistent with the pro- 
cedures set forth in the state plan for vocational education? 

6. What conclusions can be drawn? 

C-2 

1. What were the most important criteria for setting the priority of 
needs? 

2. In your opinion could other criteria be considered in developing 
the priority of needs? 

3. Are there adequate resources for the higher priority needs? 

4. Is the list of priorities consistent with the objectives for per- 
sonnel development in the state plan? 

5. What conclusions can be drawn from the data? 



C-3 

1. What were the major constraints which caused priorities to be 
lowered? 

2. What conclusions can be drawn from the data? 

D-l 

1. From which agencies were proposals submitted? 

■ 2. What type of agency submitted the largest percentage of proposals? 

3. What type of agency had the largest number funded? 
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II-6 

4. Are there other types of agencies from which proposals might have 
been solicited? 

5. What conclusions can be drawn from the data? 

D-2 

1. In your opinion were the procedures used to select proposals 
open and flexible? 

2. Might others have been used? 

3. What conclusions can be drawn? 



D-3 

1. Which criteria were most important in screening proposals for 

personnel development projects? i 

2. In your opinion, are there additional criteria which might be j 

used to screen proposals? j 

3. Was the screening process consistent with state objectives? I 

, J 

4. What conclusions can be drawn? j 
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The purpose of this section is to present (l) a definition of 
behavioral objectives, (2) guidelines to develop behavioral objectives, 

(3) examples of behavioral objectives, and (4) a suggestion to aid in 
writing difficult objectives.* 

Definition of a Behavioral Objective 

A behavioral objective is a statement which describes what a person 
or group of people will be able to do after completing a particular kind 
of instruction. It specifies: (l) the instruction, (2) the behavior or 

behaviors that are to result from that instruction, and (3) the criterion 
level the behavior must reach. Behavior is defined here as any human 
activity that can be observed by another and can be either measured or 
judged. 

Guidelines for the Development of Behavioral Objective s 

To facilitate writing behavioral objectives, it is best to divide each 
into its five components. An example of a behavioral objective and its 
five components is given below. 

The behavioral objective: 

By fiscal year 1975* state coordinators of personnel development 
and project directors who use this evaluation model will develop 
and use behavioral objectives, with the five components of each 
objective, for at least 50 percent of their objectives, as deter- 
mined by state evaluation teams. 



* Much of the materia], in this document is taken from Robert «T. Kibler 
Larry L. Barker, and David T. Miles, Behavioral Objectives and Instruction 
(Boston: Allyn and Bacon, Inc., 1970"J"I 
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The five components: 

(1) The person or persons who perform the desired behaviors (e.g., 
state coordinators of personnel development and project direc- 
tors who use this evaluation model) 

(2) The desired behavior (e.g., will develop and use) 

(3) The result or product of the behavior which will be evaluated 
(e.g., behavioral objectives, with the five components of a 
behavioral objective) 

(4) The relevant conditions under which the behavior is to occur 
(e.g., by fiscal year 1975) 

(5) The standard which will be used to Judge the result or product 
of the behavior (e.g., for at least 50 percent of their objec- 
tives, as determined by state evaluation teams) 

After writing behavioral objectives, check to make sure that each 
objective includes the five components. To prevent excessive detail in 
objectives for state programs, few objectives, perhaps 10 or less, should 
be written. 

The checklist and three examples of behavioral objectives which follow 
may aid persons writing objectives. The checklist can serve as a diagnostic 
tool to detect and correct errors in the behavioral objectives. Any 
negative answer indicates that an objective is not completely behavioral 
in terms of the definition presented here. 
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CHECK LIST 



Criteria 


YES 


NO 


1. Does each behavioral objectives emphasize a verb that 
requires action on the part of the student? 

2. Is each behavioral objective stated in terms of student 
performance (rather than teacher performance)? Does it 
describe what the learner will do when demonstrating 
this achievement of the objective? 

3. Is each behavioral objective stated so that it indicates 
terminal behavior (rather than subject matter to be 
covered during instruction)? 

4. Is each behavioral objective stated so that it includes 
only one learning outcome (rather than a combination of 
several outcomes)? 

5. Is there a sufficient number of behavioral objectives 
to adequately describe the desired achievement of the 
learners? 







t 

L 

| Examples of Behaviors] Objectives 

$ 

I The following behavioral objectives are divided into their components. 

| The number above each component refers to its numbered explanation on 



page 2. 



1. After inservice training or independent study. 



all elementary and middle 



school teachers I will use 



5 3 

at least two | career education concepts as a 



part of their curriculum, I as judged by the State Division of Vocational 



Education. 
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2. Every teacher at Metropolitan Technical Institute 



a five-minute lecture 



will present 



to nine of his peers on a topic in his area 



2 I 3 I 4 

and then privately view I the lecture I on video tape. I He will 



determine the success of his presentation hy completing a self- 



evaluation form. 



3. At a one-week seminar conducted hy the Institute on Group Relations, 



fifty community college vocational education teachers within the state 



will acquire I information about the culture of and problems encountered 



hy Blacks and Mexican-Amer leans. 



Attendance to every session and a 



score of at least 70 percent on a test devised by the instructors of 



the seminar will he accepted as successful completion of the seminar. 



Difficult Objectives 

Many times it is easier to write about behavioral objectives than 
to develop and use them in a practical situation. Kibler, Barker, and 
Miles offer the following advice for writing difficult objectives. 
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Several types of extremely important objectives are difficult 
to measure and thus, difficult to specify in "behavioral terms. 

As a matter of fact, it seems that the more significant an 
objective, the more difficult it is to measure. Examples of 
objectives which fall into the dif f icult-to- specify-and-measure 
category are those in the area of problem-solving, creativity, 
attitudes, and values. The only solution we see to this problem 
is for such objectives to be specified as clearly as possible 
and for the instructor to be as resourceful as he can in developing 
evaluative measures . . . 
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Suggested References for Writing 
Behavioral Objectives 



Kibler, Robert J. ; Barker, Larry L. , and Miles, David T. Behavioral 

Objectives and Instruction . Boston: AUyn and Bacon, Inc., 1970. 



Armstrong , Robert J. ; Cornell, Terry D.j Kroner, Robert E. , and Roberson, 

E. Wayne, eds. A Systematic Approach to Developing A Handbook 
Designed to Increase the Communication of Laymen and~ Educators . 

Tucson, Arizona: Educational Innovators Press, Inc., 1968. 

Gronlund, Norman E. Stating Behavioral Objectives for Classroom 
Instruction . New York: The MacMillan Company, 1970. 

Hernandex, David E. Writing Behavioral Objectives . New York: Barnes and 
Ncble, Inc., 1971. 

Mager, Robert F. Preparing Instructional Objectives . Palo Alto, California 
Fearon Publisher, 1962. 
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APPENDIX B 



State Personnel Development Programs Visited 
and Persons Involved in the Project 
as Consultants or Reactors 
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States Visited 
California 
Florida 
Georgia 
Illinois 
New Jersey 
New York 
Oregon 
Washington 

Advisory Committee of State Directors of Vocational Education 
Neal Andrew, New Hampshire 
Ernest Kramer, Washington 
Charles Law, North Carolina 
Jack Michie, Michigan 
George Mulling, Georgia 
Joseph Murphy, Connecticut 
John Snyder, Kansas 
Francis Tuttle, Oklahoma 
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Teacher Educators in Vocational Education 



Carl Barbell, Arizona State University 
Bernard T. Fagan, University of Kentucky 
Roger Haskill, Florida State University 
Cas Heilman, Oregon State University 
George 0* Kelly, University of Georgia 
Gordon Swanson, University of Minnesota 

Other Persons Asked to React to the Model 
Garry Bice, University of Tennessee 
Wayne Courtney, Stout State University 
Elaine House, Rutgers University 
H. C, Kazansas, University of Missouri 
Mel Miller, Oregon State University 

Consultants Used in the Project 
Conceptual Stage 

Glenn Immegart, Professor, Department of Educational 
Administration, University of Rochester 

Robert Lange, Assistant Professor, Academic Facility of 
Educational Development, The Ohio State University 

Review Stage 

Glenn Immegart 

Robert Lange 

John Struct, Director of Vocational Education, Pennsylvania 
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